It is known, from, for example BBC Research Department Report “Video noise reduction” BBC RD 1984/7 N. E. Tanton et al, July 1984, (Tanton) that random noise in a sequence of television or some other kinds of electronically generated images, e.g. scanned film, can be reduced by applying a recursive temporal filter which operates on each picture element, hereinafter abbreviated to ‘pel’. It is beneficial to reduce noise levels prior to viewing images but also prior to processes that are sensitive to the presence of noise, especially compression systems such as those defined by, but not limited to, MPEG specifications. In practice, noise control is among several important and valuable techniques employed in pre-processing elements inherent in modern compression hardware and software realisations of specifications such as MPEG.
For each pel, indexed in the image by i, an output R(i) of a recursive filter is a sum of complementary proportions of a current image C(i) and a previous resultant output image S(i) such that the proportions of C(i) and S(i) in the output R(i) are controlled by means of a fractional parameter K. If this notation is extended with a suffix ‘f’ to denote the frame it is evident that the condition Sf(i)=Rf−1(i) ensures a first order recursive temporal filter operation.
Hence, each pel in the result R(i) is expressed as:Rf(i)=K·Cf(i)+(1−K)·Sf−1(i) 0≦K≦1   Equation 1: Recursive noise reduction calculation
Where:
Cf(i) is the current input image under operation; i is the index to individual pels of that image and f is the index to the sequence of complete images,
Sf(i) is a stored array of pels equal to the result Rf−1(i) of the filter operation on the previous image and i is the index to individual pels of that image.
R(i) is the resulting pel field, which, after the operation, is copied to S(i) before the operation is performed on the next image in the sequence and i is the index to individual pels of that image.
It is expected in the filter calculations that the index i of each of R(i), C(i) and S(i) is the same so that the pels are spatially co-located in each respective image. The index of pels in each image, i, may be defined in more than one-dimension; conventionally it is convenient to use two, corresponding to the commonly-used raster scanning of television images. Thus Rf(i) becomes Rf(x,y). The index f corresponds to the frame or temporal dimension.
K is a fractional parameter that controls a proportion of the current image C(x,y) to previous image S(x,y) used to obtain the current result R(x,y). In the prior art, the parameter K does not change frequently with time, typically once per complete image, and in extreme cases may be fixed indefinitely at one value. The fractional parameter may also be operated under direct manual control. Experience shows that this is not satisfactory for some image material.
The value of K is used to control the degree of filtering (and hence noise attenuation) and that attenuation increases as K tends toward 0. At value K=0 there is no transmission of the incoming image C and the output is therefore “frozen”. This is an ideal setting to process still images where the absence of motion allows optimal filtering for noise reduction. In the presence of motion, however, the setting of K=O is, in general, far from ideal. At K=1 there is no filtering at all and the output is identically equal to the input and no noise reduction is achieved.
This technique has some limitations. Objects in motion captured in successive images cause their respective pels to change position in the image and thus their index (x,y). Motion leads to changes in the position, i.e. lateral movement, or attitude, i.e. rotational movement, of recognisable objects in the image even when there is no camera movement. Changes in object position are also caused by camera movement such as panning or tilting, or both, and changes in object position and scale caused by zooming, i.e. changes in focal length of the camera lens. In all these cases the recursive filter described above will therefore add together pels in successive images whose indices (x,y) no longer correspond to the same part of such moving objects and consequently motion will be confused with noise and the filter will reduce the value of the differences between successive images caused by motion as if it were noise. As a result, viewed images tend to blur for small amounts of movement and larger motion tends to create ghosts of any moving objects substantially compromising the image with artefacts. The essence of the problem is an inability to discriminate reliably between noise and motion, in that small amounts of motion can be difficult to distinguish from noise.
To reduce unwanted artefacts, the value of K can be defined on a per pel basis, i.e. K(x,y), from information derived from the pel C(x,y) under operation and the surrounding pels. The value of K(x,y) can be derived from the temporal difference between co-located pels, however this produces artefacts because this local difference is a function both of the sought after motion and the noise in the image. To discriminate between noise and motion requires the measurement and the subsequent exploitation of differing statistical properties of the motion and the noise.
Reinhard Kiette & Piero Zamperoni: Handbook of image processing operators. John Wiley & Sons Chichester, UK. August 1996. ISBN 0 471 95642 2 (Klette and Zamperoni) proposes a method for reducing noise in a single static image by operating dynamically across the image and producing an output image from a fraction of the “local average” of luminance pel intensity and the intensity of the collocated pel.
The fraction of the local average used is set by a function of what they describe as the “local variance”, σ. In other words the degree of filtering is determined by the amount of local activity in the image. In areas where there is such activity the degree of filtering is reduced because noise is less visible than in inactive areas where visibility is greater.
Klette and Zamperoni define their operators as follows:
                    Calculation        ⁢                                  ⁢        of        ⁢                                  ⁢        local        ⁢                                  ⁢        average                                                                                                F              _                        ⁡                          (                              r                ,                s                            )                                )                =                              1            a                    ⁢                                    ∑                              i                =                                  -                  m                                            m                        ⁢                                          ∑                                  j                  =                                      -                    n                                                  n                            ⁢                              f                ⁡                                  (                                                            r                      -                      i                                        ,                                          s                      -                      j                                                        )                                                                                        Equation        ⁢                                  ⁢        2            
This defines a local average of luminance intensity around a pel at index position (r,s) as the sum of luminance pels in an arbitrary window of 2m+1 by 2n+1 pels divided by ‘a’, the number of pels in the window and a=(2m+1).(2n+1).
Klette and Zamperoni also define σ(r,s), the local variance i.e. the mean of the squares of pel values, as:
                    Calculation        ⁢                                  ⁢        of        ⁢                                  ⁢        local        ⁢                                  ⁢        variance                                                                      σ          ⁡                      (                          r              ,              s                        )                          =                                            F              ~                        ⁡                          (                              r                ,                s                            )                                =                                    1              a                        ⁢                                          ∑                                  i                  =                                      -                    m                                                  m                            ⁢                                                ∑                                      j                    =                                          -                      n                                                        n                                ⁢                                                      [                                                                  f                        ⁡                                                  (                                                                                    r                              -                              i                                                        ,                                                          s                              -                              j                                                                                )                                                                    -                                                                        F                          _                                                ⁡                                                  (                                                      r                            ,                            s                                                    )                                                                                      ]                                    2                                                                                        Equation        ⁢                                  ⁢        3            
The operation they describe reduces noise by softening the image toward the local mean of luminance values but this causes blurring and softening as artefacts.
Klette and Zamperoni further describe that the degree of softening applied to a pel (K) is defined by,
                    Calculation        ⁢                                  ⁢        of        ⁢                                  ⁢        applied        ⁢                                  ⁢        degree        ⁢                                  ⁢        of        ⁢                                  ⁢        filtering                                                            K        =                              σ            2                                              σ              2                        +                          σ              r              2                                                          Equation        ⁢                                  ⁢        4            
Where σ is a measured local variance from mean luminance level and σr is an estimated value of the noise variance.
Expressed graphically as a surface, the calculation from Equation 4 is as illustrated in FIG. 1.
It is an object of the present invention at least to ameliorate the aforesaid shortcomings in the prior art.
It is another object of the present invention to provide means whereby discrimination between motion and noise may be improved.
It is a further object of this invention that the value of K may be varied pel-by-pel appropriately to reduce noise but also to suppress unwanted artefacts, so that K becomes K(x,y).